[PECOBLR-587] Azure Service Principal Credential Provider #621

jprakash-db · 2025-06-30T18:31:12Z

Description

This pull request introduces support for Azure Service Principal M2M (SP) authentication to the PySQL Connector

Key Changes

AzureServicePrincipalCredentialProvider, this is the provider that will deal with getting the creds using the azure credentials
ClientCredentialsTokenSource, is responsible for managing the token lifecycle for the flow where credentials are obtained using the grant_type: client_credentials
Introduced a common HTTP client called DatabricksHttpClient that can be used to unify the Http logic across the connector, to ensure standard client level behaviour

New dependencies

Introduced library PyJWT which is required for handling JWT parsing functionalities

Tests

Expanded unit tests to cover:

AzureServicePrincipalCredentialProvider.
ClientCredentialsTokenSource

Manual Testing

Tested the Azure M2M SP flow by creating a SP using Azure Entra ID and then mapping it to a workspace SP. Further queries were made to the workspace using the Azure Entra ID credentials to test it on Databricks Workspace

github-actions · 2025-06-30T18:31:22Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-01T08:19:29Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-02T18:06:43Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:14:00Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:16:31Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:18:13Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:34:21Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:37:40Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-03T05:48:26Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

pyproject.toml

src/databricks/sql/auth/auth.py

src/databricks/sql/auth/authenticators.py

src/databricks/sql/auth/common.py

src/databricks/sql/auth/oauth.py

src/databricks/sql/common/http.py

Copilot

Pull Request Overview

This PR adds support for Azure Service Principal M2M authentication to the PySQL Connector by introducing a shared HTTP client, a client-credentials token source, and a new credentials provider for Azure SP.

Introduce DatabricksHttpClient for unified HTTP logic
Add Token and ClientCredentialsTokenSource to manage OAuth client-credentials flow
Implement AzureServicePrincipalCredentialProvider and wire it through get_auth_provider

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/unit/test_thrift_field_ids.py	Formatting cleanup and consistent quote style
tests/unit/test_auth.py	Added tests for `ClientCredentialsTokenSource` and SP credential provider, JWT fixtures
src/databricks/sql/common/http.py	New singleton `DatabricksHttpClient` with retry logic
src/databricks/sql/auth/oauth.py	Introduced `Token`, `RefreshableTokenSource`, and `ClientCredentialsTokenSource`
src/databricks/sql/auth/common.py	Extended `AuthType` and helper for mapping Azure login app IDs
src/databricks/sql/auth/authenticators.py	Added `AzureServicePrincipalCredentialProvider`
src/databricks/sql/auth/auth.py	Updated `ClientContext`, `get_auth_provider`, and auth provider resolution for SP
pyproject.toml	Added `pyjwt`, moved dev dependencies under `[tool.poetry.group.dev.dependencies]`

src/databricks/sql/common/http.py

src/databricks/sql/auth/oauth.py

tests/unit/test_auth.py

src/databricks/sql/auth/auth.py

src/databricks/sql/common/http.py

samikshya-db

I don't have I have much objection with this implementation. But, what would it take for us to include dependency on SDK as it has the support for this already?

src/databricks/sql/auth/auth.py

samikshya-db · 2025-07-04T09:05:19Z

src/databricks/sql/common/http.py

@@ -0,0 +1,65 @@
+import requests


@varun-edachali-dbx has added a http client already, I suggest you two collaborate to push the common http to main first https://github.com/databricks/databricks-sql-python/blob/sea-migration/src/databricks/sql/backend/sea/utils/http_client.py

github-actions · 2025-07-07T05:21:41Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-07T06:02:13Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-07T07:48:46Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-07T08:51:54Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

github-actions · 2025-07-07T09:01:55Z

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

basic setup

5e2b792

jprakash-db self-assigned this Jun 30, 2025

jprakash-db had a problem deploying to azure-prod June 30, 2025 18:31 — with GitHub Actions Failure

Nit

05ab3e8

jprakash-db had a problem deploying to azure-prod July 1, 2025 08:19 — with GitHub Actions Failure

working

bef7ac6

jprakash-db temporarily deployed to azure-prod July 2, 2025 18:06 — with GitHub Actions Inactive

moved pyjwt to code dependency

0877b6f

jprakash-db had a problem deploying to azure-prod July 3, 2025 05:13 — with GitHub Actions Failure

nit

83d6a9e

jprakash-db had a problem deploying to azure-prod July 3, 2025 05:16 — with GitHub Actions Failure

nit

b21d015

jprakash-db had a problem deploying to azure-prod July 3, 2025 05:18 — with GitHub Actions Failure

nit

6d85d19

jprakash-db temporarily deployed to azure-prod July 3, 2025 05:34 — with GitHub Actions Inactive

nit

f518085

jprakash-db had a problem deploying to azure-prod July 3, 2025 05:37 — with GitHub Actions Failure

nit

814d1cb

jprakash-db had a problem deploying to azure-prod July 3, 2025 05:47 — with GitHub Actions Failure

nit

ce4543c

jprakash-db temporarily deployed to azure-prod July 3, 2025 05:51 — with GitHub Actions Inactive

jprakash-db changed the title ~~Azure Service Principal Credential Provider~~ [PECOBLR-587] Azure Service Principal Credential Provider Jul 3, 2025

jprakash-db removed request for jackyhu-db, madhav-db, jayantsing-db and deeksha-db July 4, 2025 04:53

vikrantpuppala reviewed Jul 4, 2025

View reviewed changes

vikrantpuppala requested a review from Copilot July 4, 2025 05:28

Copilot AI reviewed Jul 4, 2025

View reviewed changes

src/databricks/sql/common/http.py Outdated Show resolved Hide resolved

src/databricks/sql/auth/oauth.py Outdated Show resolved Hide resolved

tests/unit/test_auth.py Outdated Show resolved Hide resolved

src/databricks/sql/auth/auth.py Outdated Show resolved Hide resolved

gopalldb reviewed Jul 4, 2025

View reviewed changes

src/databricks/sql/auth/auth.py Show resolved Hide resolved

gopalldb reviewed Jul 4, 2025

View reviewed changes

src/databricks/sql/auth/auth.py Show resolved Hide resolved

gopalldb reviewed Jul 4, 2025

View reviewed changes

src/databricks/sql/common/http.py Show resolved Hide resolved

samikshya-db reviewed Jul 4, 2025

View reviewed changes

jprakash-db added 2 commits July 5, 2025 14:31

testing sdk

42841f1

Refractor

a377ce7

jprakash-db had a problem deploying to azure-prod July 7, 2025 05:21 — with GitHub Actions Failure

logging

bf5c565

jprakash-db temporarily deployed to azure-prod July 7, 2025 06:02 — with GitHub Actions Inactive

vikrantpuppala mentioned this pull request Jul 7, 2025

Add optional telemetry support to the python connector #628

Open

8 tasks

nit

3f7d476

jprakash-db temporarily deployed to azure-prod July 7, 2025 07:48 — with GitHub Actions Inactive

nit

601b08e

jprakash-db temporarily deployed to azure-prod July 7, 2025 08:51 — with GitHub Actions Inactive

nit

9a948eb

jprakash-db had a problem deploying to azure-prod July 7, 2025 09:01 — with GitHub Actions Failure

jprakash-db temporarily deployed to azure-prod July 7, 2025 10:59 — with GitHub Actions Inactive

[PECOBLR-587] Azure Service Principal Credential Provider #621

Are you sure you want to change the base?

[PECOBLR-587] Azure Service Principal Credential Provider #621

Uh oh!

Conversation

jprakash-db commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key Changes

New dependencies

Tests

Uh oh!

github-actions bot commented Jun 30, 2025

Uh oh!

github-actions bot commented Jul 1, 2025

Uh oh!

github-actions bot commented Jul 2, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samikshya-db left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

samikshya-db Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

Uh oh!

jprakash-db commented Jun 30, 2025 •

edited

Loading